A single-sensor digital camera employs a color filter array (CFA) in order to capture full-color information from a single two dimensional array of light-sensitive pixels. The CFA consists of an array of color filters that filter the light being detected by each pixel. As a result, each pixel receives light from only one color, or in the case of a panchromatic or “clear” filter, from all colors. In order to reproduce a full-color image from the CFA image, three color values must be produced at each pixel location. This is accomplished by interpolating the missing color values from neighboring pixel values.
The best known CFA pattern uses three colors channels as described by Bayer (U.S. Pat. No. 3,971,065) and shown in FIG. 2. The Bayer CFA has three color channels which enables full-color reproduction capability. However, the exact spectral responsivities (“colors”) of the three channels represent a compromise. In order to improve color fidelity and broaden the range of colors that can be captured by the CFA (i.e., the color gamut), the spectral responsivities need to be made more selective (“narrowed”). This has the side effect of reducing the overall amount of light that reaches the pixel and, therefore, reduces its sensitivity to light. As a consequence, the pixel value becomes more susceptible to noise from non-imaging sources, e.g., thermal noise. One solution to the noise problem is to make the CFA spectral responsivities less selective (“broader”) to increase the overall amount of light that reaches the pixel. However, this comes with the side effect of reducing color fidelity.
A solution to this three-channel CFA limitation is to employ a four-channel CFA composed of three colors with “narrow” spectral sensitivities and one color with a “broad” spectral sensitivity. The “broadest” such channel would be panchromatic or “clear” which would be sensitive to the full spectrum of light. The three “narrowband” color channels would produce an image with higher color fidelity and lower spatial resolution while the fourth “broadband” panchromatic channel would produce an image with lower noise and higher spatial resolution. These high color fidelity, low spatial resolution and low noise, high spatial resolution images would then be merged into a final high color fidelity, low noise, high spatial resolution image.
In order to produce a high spatial resolution panchromatic image while maintaining high color fidelity from the color pixels, the number and arrangement of panchromatic pixels within the CFA and the corresponding interpolation algorithms must be properly chosen. There are a variety of examples in the prior art with one or more liabilities in this regard. Frame (U.S. Pat. No. 7,012,643) teaches a CFA as shown in FIG. 19 that has only a single red (R), green (G), and blue (B) pixel within a 9×9 square of panchromatic (P) pixels. The problem with Frame is that the resulting color spatial resolution is too low to produce all but the lowest frequency color details in the image.
Yamagami et al. (U.S. Pat. No. 5,323,233) describe two CFA patterns as shown in FIGS. 20A and 20B that have equal amounts of panchromatic and color pixels, avoiding the liability of Frame. Yamagami et al. go on to teach using simple bilinear interpolation as the means for interpolating the missing panchromatic values. The use of solely linear interpolation methods (such as bilinear interpolation) strongly limits the spatial resolution of the interpolated image. Nonlinear methods, such as that described in Adams et al. (U.S. Pat. No. 5,506,619), produce higher spatial resolution interpolated images, provided the CFA pattern permits their use. FIG. 21A illustrates the pattern used in Adams et al. Green (G) pixels, which provide the high spatial frequency resolution in the three channel system shown in FIG. 2, alternate with color (C) pixels in both the horizontal and vertical directions about a central color pixel. It is important to note that these color pixels are all the same color, e.g., red pixels. FIG. 21B shows a similar pattern that uses panchromatic (P) pixels in place of green pixels. It should be noted at this point that for a four-channel system it is not possible to arrange all four channels (R, G, B, and P) in such a way that the pattern shown in FIG. 21B occurs at all color (R, G, B) pixel locations across the sensor. Therefore, any possible arrangement will be some compromise in this manner. With regard to Yamagami et al. FIG. 20A has green and panchromatic pixels arranged as in FIG. 21B, but the red and blue pixels are not so arranged. After FIG. 21B, an arrangement such as in FIG. 21C is preferred, but FIG. 20A does not have this either with respect to the red or blue pixels. FIG. 20B does not have the patterns of FIG. 21B or FIG. 21C for any color pixels. Tanaka et al. (U.S. Pat. No. 4,437,112) describe a number of CFA patterns of which the most relevant one to this discussion is FIG. 22. In FIG. 22 cyan (C), yellow (Y), green (G), and panchromatic (P) pixels are arranged so that the green pixels are surrounded by the neighborhood shown in FIG. 21C. However, the yellow and cyan pixels do not conform to the patterns of either FIG. 21B or FIG. 21C. The same difficulties exist with the other patterns taught by Tanaka et al.
Hamilton et al. (U.S. Pat. Appl. No. 2007/0024879) teach a large number of CFA patterns of which two are shown in FIGS. 23A and 23B. The liabilities of these, as well as all of the other patterns disclosed in Hamilton et al., are the lack of FIG. 21B and FIG. 21C pixel arrangements.
Kijima et al. (U.S. Pat. Appl. No. 2007/0177236) describe a large number of CFA patterns of which the most relevant CFA pattern is shown in FIG. 24. While the double row of panchromatic pixels provides a FIG. 21C arrangement in the vertical direction, no such horizontal arrangement of side-by-side panchromatic values exists in FIG. 24.
Thus, there exists a need for a four-channel CFA pattern with three narrowband color channels and one broadband panchromatic channel with enough color pixels to provide sufficient color spatial resolution and arranged in such a way as to permit the effective nonlinear interpolation of the missing panchromatic values.